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Following the approach in our earlier paper and using the gradient estimates 
developed in [§] and , we give another Liouville type theorem for some conformally 
invariant fully nonlinear equations. Various Liouville type theorems for conformally 
invariant equations have been obtained by Obata, Gidas-Ni-Nirenberg, Caffarelli- 
Gidas-Spruck, Viaclovsky, Chang-Gursky-Yang, and Li-Li. For these, as well as for 
related works, see and the references therein. 

For n > 3, let 5"^" be the set of nxn real symmetric matrices, S!l^"' C 5"^" be 
the set of positive definite matrices, and let 0{n) be the set of nx n real orthogonal 
matrices. 

For 1 < A; < r?,, let 

f^fc(A) = J2 K---K^ A= (Ai,---,A„) GR", 

l<ii<-<ife<n 

denote the k—th symmetric function, and let denote the connected component 
of {A e M" I crfc(A) > 0} containing the positive cone {A G M" | Ai, ■ ■ ■ , A„ > 0}. It 
is known that 

F, = {A G I Ai, ■ ■ ■ , A„ > 0}, Fi = {A G I Ai + ■ ■ ■ + A, > 0}, 
Ffe = {A G I ai(A) > 0, ■ ■ ■ , afc(A) > 0}, 
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Tk is a convex cone with its vertex at the origin with the properties 

r„c---cr2cri, 

dak 



> in Ffe, 1 < i < n, 



cr^ is concave in T^- 
For a positive function let 

A" := -M n-2V M+7 -rrrli "-2Vw(8)Vw-7 -rrrli ""2 Vwr/, 

n — 2 (n — 2)^ (n — 2)^ 

where / is the n x n identity matrix. 

Assume U C 5"^" is an open set satisfying 

Q-^UO = U, V O e 0{n), 

and 

Un{M + tN \0 <t< 00} is convex V M e 5"^", N e 5^^", 

and 

n + 1 

Fu :^ {X{M)\ M eU} GFk, for some > 

where A(M) denotes the eigenvalues of M. 
Let F e C2(C/) satisfy 

F{q-^MO) = F{M), y M eU, O e 0(n), 

does not belong to F~^{1), 
{Fij{M)) > 0, V M e [/, 
F is locally concave in U, 

and, for some < 7 < 1, 

n 1 

^ F,j{M)M,, < -|M|i-^^F,,(M), V M e C/, F(M) = 1, |M| > 1, 

i.j=l i=l 

Where F,,(Af) := ^(M). 

We estabhsh in this paper the following Liouville type theorem. 
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Theorem 1 For n > 3, let U G 5"^" be an open set satisfying ([^, and (Qj, 
and let F G C\U) satisfy ^, ^ and (gj. Let u G C^(R") be a positive 

solution of 

F(v4") = 1, A" G U, on R". 

Then for some x G M", and some positive constants a and b satisfying 26^a~^/ G [/ 
and F{2b^a-^I) = 1, 



n-2 
2 



Remark 1 In Theorem^, if F is in C'^'^iU) for some {3 G (0,1), then, since the 
equation is elliptic, any positive solution u is in fact in C^'f^ . 

We give a consequence of Theorem |l[ 
Let 

r C R" be an open convex cone with its vertex at the origin (10) 

such that 

n + 1 

r„ C r C Ffc, for some k > — - — , (11) 

and 

r is symmetric in the Aj. (12) 

Let 

/ G C^(r) n C°(r) be concave and symmetric in the Aj. (13) 
In addition, we assume that 

/ = on ar; /a, > on F V 1 < i < n, (14) 

and 

Jim /(sA) = cx), VAgF. (15) 
By ( |T4|) and ([T5|) , there exists a unique b > such that 

f{be) = 1, (16) 



where e = (1, ■ ■ ■ , 1) 
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Corollary 1 For n > 3, let (/, T) satisfy 1^, (jg;, 1^, 1^, 1^ and and 

let u G C'^(]R") he a positive solution of 

/(A(A")) = 1, A(A") G r, on K". 
Then for some x G M", and some positive constant a, 



n — 2 
2 



1 + |a^6|x — 



V x G 



Proof of Theorem |I]. Since implies the superharmonicity of the positive func- 
tion u on M", we have liminf > 0. Let w{x) = |^|n-2 '^( j^) for x G R"\{0}. 

Then w is regular at oo, liminf w(x) > 0, and w satisfies 

\x\^0 

F{A^) = 1, A^eU, onR'^VjO}. 

9 2 

Let (^(x) = ^^^iy(x) "-2. Then, for some positive constant Ci, 

0<e<Ci onS2\{0}. (17) 

By (D and lemma 6.3 in §, X{D^^{x)) G for x G \ {0}. Let P be any 
hyperplane which intersects Bi but does not pass through the origin, and let C,p be 
the restriction of ^ on P. Then 

A(D2ep) G Ffc^i C M"-\ onPnSs, 

where D^^p denotes — 1) x (n — 1) Hessian of C,p, and A(D^^p) denotes the 
eigenvalues of D^^p. Here we have used the following property of F^: If A(M) G 
Ffc C M", then A(M) G F^^i C M""^ where Mij = Mij for 1 < z, j < n - 1. Since 
k > we have k — 1 > As in [||, by using theorem 2.7 in we have, for 
some constants a G (0, 1) (depending only on n and k) and C > (depending only 
on n, k and Ci), that 

UWcHPnB,) < C. (18) 

For any x,y & Bi\ {0}, we pick Zi G M" such that Zi and the line going 
through X and y + Zi does not go through the origin. Then x and ?/ + -Zj lies on some 
hyperplane Pi which does not go through the origin. Thus, by ([T8|), 



\^^^)^^^y + ,^)\<C\x-{y + z,)\ 
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for some constant C depending only on n, k and Ci. Sending i to infinity, we have 

\ax)-ay)\<c\x-y)r. 

Therefore ^ can be extended to a function in C"{Bi). 

We distinguish into two cases. 
Case 1. ^(0) = 0. 
Case 2. ^(0) > 0. 

In Case 1, \im\x\^c>o{\x\"'^'^u{x)) = oo. For every x G M", as in the proof of lemma 
2.1 in 0, there exists Xo{x) > such that 

u^Ay) := i-^r-M^ + ^j^^^) < u{y), V < A < Ao(x), \y - x\ > A. 

\y ~ ^\ \y ~ x\ 

Set, for x e R", 

A(x) = sup{/i > I u^ xiy) < '^iy)^ for all I?/ — x| > A, < A < /i}. 



Lemma 1 A(x) = oo for all x G M". 

Proof of Lemma |I|. If X{x) < oo for some x G M". Making a translation, we may as- 
sume without loss of generality that x = 0, and we still have \im\^\^oo{\x\"'~^u{x)) = 
oo. Thus, there exists some R > X + 9 (we use notation A = A(0) such that 

ux{y)<u{y), V0< A< A + 2, >i?, (19) 

where we have used notation u\ = Mo,a- 
By the definition of A, 

u-xiy) < u{y), V \y\ > X. 
Let Wt := tu + {1 — t)ux, < t < 1. Then, as in the proof of lemma 2.1 in [Q], 

L{u - ux) = 0, in M" \ Bx, 

where 

L = aij{y)dij + hi{y)di + c(y), 
and foj and c are continuous functions. 
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Using the Hopf Lemma and the strong maximum principle as in the proof of 
lemma 2.1 in 0, we have 



and 

din - u,) ^ 



dr 



d_ 

3r 

The following argument is similar to the one used in the proof of lemma 2.2 in 



where denotes the outer normal differentiation. 

or 



has a positive lower bound. Using the 



d]. Since dBx is compact, ^^^q^^^ 
regularity of ti, we can find some Q < 6 <1 such that 

d{u-ux) ,. ^ VA< A< A + 5,A< kjl < A + 5. 

or 

Since {u — u\){y) = for |y| = A, the above implies 

ux{y)<u{y), VA < A < A + 5,A < |?/| < A + 5. 

Since {u\ — u){y) < for A + 5 < \y\ < R, and since the set is compact, there 
exists e G (0, 6) such that 

uxiy)<u{y), W X< \<\ + e,X + S <\y\< R. 

Here we have used the the continuity of u. 
We have proved, for the e above, that 

ux{y)<u{y), V A< A< A + e, \y\ > \. 

This violates the definition of A. Lemma |I| is established. 

□ 

It follows from Lemma |T| that 

Ux,\iy) < u{y), V X e M", < A < cx), - x| > A. 

This, together with some calculus lemma (see, e.g., lemma 11.2 in [Q), implies that 
M is a constant on K", thus = 0. This is impossible because of (^. We have 
ruled out Case 1. 

In Case 2, there exists some constant < 6 < ^ such that 

6<w<\, on Bios- (20) 
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Lemma 2 

limsup(|x||Vw(x)|) < oo. 

Proof of Lemma |2|. For any < r < 56, let v{y) := w{ry) for < ||/| < 2. Then 
V satisfies 

F{r-^A') = l, A'^eU, onS2\{0}. (21) 

For any x G -Bs \53 , as in the proof of lemma 2.1 in 0], there exists Xo{x) G (0, |) 
such that 

v.Av) ■= (^^r-'^(^+ ^|'^^~|2^ ) < v{y), Wye {B2\B,)\B,{x), < A < Xo{x). 
\y — x\ \y — x\ 2 

Set, for X E BsX Bi, 

2 4 

~X{x) = snp{fi > I v,,xiy) < v{y), V y G (^2 \ Bi) \ Bxix), < A < /x}. 

Using the Hopf Lemma and the strong maximum principle as in the proof of lemma 
2.1 in p|, and using the argument in Lemma [l|, we know that stopping at A(x) 
is due to a boundary touching, i.e., there exists some yQ G d{B2 \ Bi) such that 

VxX^)(yo) = ^(?/o), i-e. 

/ A(x) .„ , , X^riyQ — x). 

\yo-x\ \yo-x\^ 

from which we deduce, using (pOf ), that 

wirx H — ^ 12 ) 



Thus we have shown that for any x E B'i\ Bs and any < A < we have 

^^x,a(2/) < v{y), V y G ^2 \ fii , I?/ - x| > A. 

This and some calculus lemma (see lemma 1 in 0) imply, for some constant C 
depending only on 6, that 

\Vv{y)\ < Cv{y) V |y| = 1, 
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\Vw{ry)\<C^^, V|y| = l. 
r 

Since this holds for all < r < 55, we have 

\z\\Ww{z)\<Cw{z), V0<|2|<5(5. 
Lemma ^ is established. 

□ 

Our next lemma provides estimates of the second derivatives of w near the origin. 
Lemma 3 

limsup(|a;p|V^ty(a;)|) < oo. 
|xHo 



Proof of Lemma |3|. Let 5 be as in the proof of Lemma 0, < r < 55, and 
v{y) := w{ry). Then v satisfies (^If), i.e., 

F{A')=r\ A^eU, on52\{0}, 

where U := r'^U and F{M) := r^F{r-^M),M e U. Clearly, (F, U) satisfies (H), (D, 
(§),(§, ®, (0) (with {F,U) replaced by {F.U)) , and 

^ F,,{M)M,, < -|M|i-^^F,,(M), V M G ^, F(M) = r^, \M\ > 1. 

ij=l i=l 



We know from ( pof ) and Lemma g that 

t;+|Vi;|<C on B3\B 



for some constant C independent of r. 

Following, with minor modification, the computation in the proof of theorem 1.6 
in [0 (with F there replaced by our F, v there replaced by logv with our v, 
and keep in mind that h there is a constant r^; for some earlier works on second 
derivative estimates, see remark 1.13 in 0), we obtain 

< C on dBi 

for some constant C independent of r. Lemma 13 follows immediately. 
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□ 

Since w G C"(i?i) and since we have proved that 

hmsup(|x||Vw(x)| + V^tL'(x)l) < oo, 

\x\^0 

we can apply lemma 6.4 in to obtain limsupi^.i^od^l"'^"^ l^"^!^)!) < 
particular, 

lim (|x||Viy(2;)|) = 0. 

|a:|^0 

Now we are in a position to apply theorem 1.2 in (with mq.i there being our 
w) to conclude that u must be of the form (|^). Theorem ^ is established. 

□ 

Proof of Corollary Let 

U:={Me 5"^" I A(M) G r}, 

and 

F(M) := /(A(M)), M eU. 

To establish Corollary 0, we only need to verify that (F, U) satisfies the hypothesis 
of Theorem |I|. These are well known to people in the field, but for convenience of 
the reader, we provide some details. Since T is an open subset of M", U is an open 
subset of jS""^". Since orthogonal conjugation does not change the set of eigenvalues 
and since F is symmetric in the Aj, we know that U satisfies (p and F satisfies (^. 
Since r„ C F and T satisfies we know that A + /i = 2(^i^) G T for all A G F 
and /i G r„. For M G [/ and G 5"^", let A„(M) > ■ ■ ■ > Ai(M) denote the 
eigenvalues of M, we know that 

Xi{M) = inf sup (x'Mx), 1 < z < n. 

dim K=i^^x,\\x\\=l 

Similar formula holds for M + N. Thus Ai(M + A^) > Ai(M) for all 1 < i < Write 
A = (Ai(M), • ■ • , Xn{M)) and ^ = (Ai(M + iV) - Ai(M), ■ • • , A,(M + N) - A„(M)), 
then A G F and /i G F„, thus \ + = (Ai(M + A^), ■ ■ ■ , A„(M + A^)) G F, i.e. 
M + N eU. So [/ satisfies (|). Since Tu = F, (D follows from Clearly, (|) 

follows from /(O) = 0. Property and (|^ can be deduced from the concavity of 
/ in F and the fact that /a, > in F for every 1 < i < n, see e.g., For all A G F 
satisfying /(A) = 1, we have, using the concavity of / in F and the convexity of F, 



1 = fibe) < /(A) + ^ -X,) = 1 + Y: /a. (A) (6 - A.), 

i i 
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I.e., 



E/a.(A)A.<65:/a.(A). 



This, after diagonalizing M by an orthogonal conjugation, imphes (^). Corollary 
is established. 



□ 
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